Working with Notebooks

You should already have Jupyter notebooks installed or remotely accessible...

Start the notebook server

open the notebook homepage

and create a new notebook.

Code Cells

A new notebook contains a single, executable code cell.

Add a line of code and execute it by:

  • clicking the run button, or
  • click in the cell, and press shift-return

In [2]:
print('hello world')


hello world

Adding Narrative - Markdown Cells

Add a new cell to the notebook: click the + button on the toolbar

Change the cell type using the drop down list in the toolbar or by using the ESC-M keyboard shortcut.

To "open" or select a markdown cell for editing, double click the cell.

View the rendered markdown by running the cell:

  • hit the play button on the toolbar, or
  • use the SHIFT-RETURN keyboard shortcut.

Simple Markdown 1

Markdown cells use markdown to generate formatted text:

  • inline styles
    • emphasise text
    • strongly emphasise text

Sub-headings

Markdown can also show inline code styles as well as code blocks:

def mycode():
    ''' Here is my non-executable code '''
    pass

So what does the actual markdown look like?


In [ ]:
# Simple Markdown 1

Markdown cells use markdown to generate formatted text:

- inline styles
  - *emphasise text*
  - __strongly emphasise__ text 
  
## Sub-headings
Markdown can also show `inline code` styles as well as code blocks:

````
def mycode():
    ''' Here is my non-executable code '''
    pass
````

Simple Markdown 2

Markdown can include weblinks, eg to Data Carpentry, as well as links to named elements in the same notebook, or other notebooks.

Markdown can embed images:

Jupyter logo

So how do we do that?


In [ ]:
<a name=""></a>
# Simple Markdown 2

Markdown can include weblinks, eg to [Data Carpentry](https://datacarpentry.org), as well as links to named elements [in the same notebook](#navlink), or [other notebooks](path/example.ipynb#exampleNavlink).

Markdown can embed images:

[Jupyter logo](./jupyter-logo.png)

Markdown cells can include Latex Expressions

Mathematical expessions can be rendered inline by wrapping a LaTeX expression (no spaces) with a $ either side.

$e^x=\sum_{i=0}^\infty \frac{1}{i!}x^i$

is rendered inline: $e^x=\sum_{i=0}^\infty \frac{1}{i!}x^i$

Use $$ to render in the centre of a new line: $$e^x=\sum_{i=0}^\infty \frac{1}{i!}x^i$$

Navigating and Selecting Cells

To select a cell, click on it. The selected cell will be surrounded by a box with the left hand side highlighted.

Move the selection focus to the cell above/below using the keyboard up/down arrow keys.

Select multiple adjacent cells using SHIFT-UP ARROW or SHIFT-DOWN ARROW

Managing Cells - Add, Delete

Add a new cell to the notebook by:

  • click the + button on the toolbar
  • Insert -> Insert Cell Above or ESC-A
  • Insert -> Insert Cell Below or ESC-B

Delete a cell by selecting it and:

  • click the scissors button on the toolbar
  • Edit -> Delete cells or ESC-X

Undelete the last deleted cell:

  • Edit -> Undo Delete cells or ESC-Z

Managing Cells - Reorder

Reorder cells by:

  • moving them up and down the notebook using the up and down arrows on the toolbar
  • Edit -> Move Cell Up or Edit -> Move Cell Down
  • cutting and pasting them:
    • Edit - >Cut or Edit->Paste Cells Above or Edit->Paste Cells Below
    • on the toolbar, Cut selected cells then Paste selected cells

You can also copy selected cells from the toolbar, Edit -> Copy Cells or ESC-C.

Managing Cells - Merging and Splitting

Splitting overlong cells: Edit -> Split Cell

Merging adjacent cells: Edit -> Merge Cell Above or Edit -> Merge Cell Below.

Cell outputs

If the last line of code produces an output, the output will be embedded in the notebook below the code cell:


In [4]:
a=1
b=2

a+b


Out[4]:
3

We Can Run a Cell Multiple Times

Each time the cell us run, the state of the underlying python process is updated, even if the visual display of other cells in the notebook is not.


In [5]:
print(a)


1

In [8]:
#Run this cell multiple times
a=a+1
a


Out[8]:
4

Working code cells harder

We can import packages as you might expect:


In [13]:
import numpy as np

np.pi


Out[13]:
3.141592653589793

Code cells also act as a commandline prompt - prefix with a !


In [11]:
! ls *.ipynb # Linux / Mac;
#for windows: ! dir *.ipynb


Ecosystem elements.ipynb
Intro-to-reproducible-research.ipynb
Navigating the notebook - instructor script.ipynb
Resources.ipynb
Workshop slides - using the notebooks.ipynb
getting_started_with_jupyter_notebooks.ipynb

Line numbering in code cells can be toggled with ESC-L.

IPython Cell Magics

  • %matplotlib inline: enable inline display of matplotlib generated graphics
  • %whos: display a list of variables and their values as set in the kernel
  • %env: display a list of environment variables and their current values in the host environment.

Code cells can produce rich output too

Code cell outputs can render tables and charts:


In [16]:
import pandas as pd

pd.DataFrame({'col1':['x','y'],'col2':[1,2]})


Out[16]:
col1 col2
0 x 1
1 y 2

In [14]:
%matplotlib inline

import matplotlib.pyplot as plt

# Create 1000 evenly-spaced values from 0 to 2 pi
x = np.linspace(0, 2*np.pi, 1000)  

#Plot a sine wave over those values
y = np.sin(x)

plt.plot(x, y)

#You can prevent the display of object details returned from the plot by:
## - adding a semi-colon (;) at the end of the final statement


Out[14]:
[<matplotlib.lines.Line2D at 0x1132de780>]

Notebooks Can be Support Interactive Widgets that Help You Explore a Dataset

If you reate a function that accepts one or more parameters, you may be able to use it as the basis of an automatically generated application.

For example, suppose we have a function that will plot a sine wave over the range 0..2 pi for a specified frequency, passed into the function as a parameter:


In [18]:
#If no frequency value is specified, use the default setting: f=1

def sinplot(f=1):
    #Define a range of x values
    x = np.linspace(0, 2*np.pi, 1000)  

    #Plot a sine wave with the specified frequency over that range
    y = np.sin(f*x)

    #Plot the chart
    plt.plot(x, y)
    
sinplot(f=3)


Using ipywidgets interact()

Pass the name of your function, and the default values of the parameters, to the ipywidgets interact() function to automatically create interactive widgets to control the parameter values.


In [20]:
from ipywidgets import interact

interact(sinplot, f=5)


Out[20]:
<function __main__.sinplot>

You can also specify the range of values Applied to an interact() slider:

interact(sinplot, f=[0,20])

Or the range of values and the step size:

interact(sinplot, f=[0,20,5])

Checking Reproducibility

Clear the output of a selected cell: Cell -> Current Output -> Clear

Clear the output of all cells in the notebook: Cell -> All Output -> Clear

Note the the state of the underlying kernel will not be affected - only the rendered display in the notebook.

Run multiple cells:

  • Cells -> Run All Above
  • Cells -> Run All Below
  • Cells -> Run All

Run cells from scratch (i.e. from a fresh kernel), Kernel -> Restart and Clear Output and then run the cells you want.

To run all the cells in the notebook from scratch: Kernel -> Restart and Run All

Troubleshooting

Tips and tricks for when it goes wrong...

Getting Help

Find help files for the notebooks from the Help menu.

Display keyboard shortcuts using Help -> Keyboard Shortcuts or ESC-H.

Code cells support autocomplete: start typing and then TAB to see what options are available...

Access documentation for a function - add a ? and run the cell:


In [ ]:
pd.DataFrame?

Saving, Checkpointing and Reverting the Notebook

The notebook wil autosave every few minutes.

You can also create a checkpoint using the floppy/save icon on the toolbar or File -> Save and Checkpoint.

You can revert the notebook to a saved checkpoint using File -> Revert to Saved Checkpoint.

Permanently Running Cells

Code cells that are running (or queued for running) display an asterisk in the cell In [] indicator.

To stop execution of a running cell (and prevent queued cells from executing):

  • press the stop button on the toolbar
  • Kernel -> Interrupt

If the notebook is still hanging, you may need to restart the kernel: Kernel -> Restart

And finally...

This slide deck was produced from a notebook styled as a slideeck:

  • View -> Cell Toolbar -> Slideshow, and then either:
  • jupyter nbconvert slideTest.ipynb --to slides --post serve, or
  • jupyter nbconvert slideTest.ipynb --to slides && python -m SimpleHTTPServer 8000 #then go to localhost:8000 in browser